64 research outputs found
LTT-GAN: Looking Through Turbulence by Inverting GANs
In many applications of long-range imaging, we are faced with a scenario
where a person appearing in the captured imagery is often degraded by
atmospheric turbulence. However, restoring such degraded images for face
verification is difficult since the degradation causes images to be
geometrically distorted and blurry. To mitigate the turbulence effect, in this
paper, we propose the first turbulence mitigation method that makes use of
visual priors encapsulated by a well-trained GAN. Based on the visual priors,
we propose to learn to preserve the identity of restored images on a spatial
periodic contextual distance. Such a distance can keep the realism of restored
images from the GAN while considering the identity difference at the network
learning. In addition, hierarchical pseudo connections are proposed for
facilitating the identity-preserving learning by introducing more appearance
variance without identity changing. Extensive experiments show that our method
significantly outperforms prior art in both the visual quality and face
verification accuracy of restored results.Comment: Project Page: https://kfmei.page/LTT-GAN
AttaNet: Attention-Augmented Network for Fast and Accurate Scene Parsing
Two factors have proven to be very important to the performance of semantic
segmentation models: global context and multi-level semantics. However,
generating features that capture both factors always leads to high
computational complexity, which is problematic in real-time scenarios. In this
paper, we propose a new model, called Attention-Augmented Network (AttaNet), to
capture both global context and multilevel semantics while keeping the
efficiency high. AttaNet consists of two primary modules: Strip Attention
Module (SAM) and Attention Fusion Module (AFM). Viewing that in challenging
images with low segmentation accuracy, there are a significantly larger amount
of vertical strip areas than horizontal ones, SAM utilizes a striping operation
to reduce the complexity of encoding global context in the vertical direction
drastically while keeping most of contextual information, compared to the
non-local approaches. Moreover, AFM follows a cross-level aggregation strategy
to limit the computation, and adopts an attention strategy to weight the
importance of different levels of features at each pixel when fusing them,
obtaining an efficient multi-level representation. We have conducted extensive
experiments on two semantic segmentation benchmarks, and our network achieves
different levels of speed/accuracy trade-offs on Cityscapes, e.g., 71 FPS/79.9%
mIoU, 130 FPS/78.5% mIoU, and 180 FPS/70.1% mIoU, and leading performance on
ADE20K as well.Comment: AAAI 202
Deep Semantic Statistics Matching (D2SM) Denoising Network
The ultimate aim of image restoration like denoising is to find an exact
correlation between the noisy and clear image domains. But the optimization of
end-to-end denoising learning like pixel-wise losses is performed in a
sample-to-sample manner, which ignores the intrinsic correlation of images,
especially semantics. In this paper, we introduce the Deep Semantic Statistics
Matching (D2SM) Denoising Network. It exploits semantic features of pretrained
classification networks, then it implicitly matches the probabilistic
distribution of clear images at the semantic feature space. By learning to
preserve the semantic distribution of denoised images, we empirically find our
method significantly improves the denoising capabilities of networks, and the
denoised results can be better understood by high-level vision tasks.
Comprehensive experiments conducted on the noisy Cityscapes dataset demonstrate
the superiority of our method on both the denoising performance and semantic
segmentation accuracy. Moreover, the performance improvement observed on our
extended tasks including super-resolution and dehazing experiments shows its
potentiality as a new general plug-and-play component.Comment: ECCV2022, for Project Page, see https://kfmei.page/d2sm
AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models
Although many long-range imaging systems are designed to support extended
vision applications, a natural obstacle to their operation is degradation due
to atmospheric turbulence. Atmospheric turbulence causes significant
degradation to image quality by introducing blur and geometric distortion. In
recent years, various deep learning-based single image atmospheric turbulence
mitigation methods, including CNN-based and GAN inversion-based, have been
proposed in the literature which attempt to remove the distortion in the image.
However, some of these methods are difficult to train and often fail to
reconstruct facial features and produce unrealistic results especially in the
case of high turbulence. Denoising Diffusion Probabilistic Models (DDPMs) have
recently gained some traction because of their stable training process and
their ability to generate high quality images. In this paper, we propose the
first DDPM-based solution for the problem of atmospheric turbulence mitigation.
We also propose a fast sampling technique for reducing the inference times for
conditional DDPMs. Extensive experiments are conducted on synthetic and
real-world data to show the significance of our model. To facilitate further
research, all codes and pretrained models are publically available at
http://github.com/Nithin-GK/AT-DDPMComment: Accepted to IEEE WACV 202
- …